How AI should be used in radiology: assessing ambiguity and completeness of intended use statements of commercial AI products.

BACKGROUND: Intended use statements (IUSs) are mandatory to obtain regulatory clearance for artificial intelligence (AI)-based medical devices in the European Union. In order to guide the safe use of AI-based medical devices, IUSs need to contain comprehensive and understandable information. This study analyzes the IUSs of CE-marked AI products listed on AIforRadiology.com for ambiguity and completeness. METHODS: We retrieved 157 IUSs of CE-marked AI products listed on AIforRadiology.com in September 2022. Duplicate products (n = 1), discontinued products (n = 3), and duplicate statements (n = 14) were excluded. The resulting IUSs were assessed for the presence of 6 items: medical indication, part of the body, patient population, user profile, use environment, and operating principle. Disclaimers, defined as contra-indications or warnings in the IUS, were identified and compared with claims. RESULTS: Of 139 AI products, the majority (n = 78) of IUSs mentioned 3 or less items. IUSs of only 7 products mentioned all 6 items. The intended body part (n = 115) and the operating principle (n = 116) were the most frequently mentioned components, while the intended use environment (n = 24) and intended patient population (n = 29) were mentioned less frequently. Fifty-six statements contained disclaimers that conflicted with the claims in 13 cases. CONCLUSION: The majority of IUSs of CE-marked AI-based medical devices lack substantial information and, in few cases, contradict the claims of the product. CRITICAL RELEVANCE STATEMENT: To ensure correct usage and to avoid off-label use or foreseeable misuse of AI-based medical devices in radiology, manufacturers are encouraged to provide more comprehensive and less ambiguous intended use statements. KEY POINTS: • Radiologists must know AI products' intended use to avoid off-label use or misuse. • Ninety-five percent (n = 132/139) of the intended use statements analyzed were incomplete. • Nine percent (n = 13) of the intended use statements held disclaimers contradicting the claim of the AI product. • Manufacturers and regulatory bodies must ensure that intended use statements are comprehensive.

Approval, Evidence, and "Off-Label" Device Utilization: The Patent Foramen Ovale Closure Story.

BACKGROUND: Following regulatory approval, medical devices may be used "off-label." Patent foramen ovale (PFO) closure is indicated to reduce recurrent stroke but has been proposed for other indications, including migraine, transient ischemic attack, and diving decompression illness. We sought to evaluate PFO closure rates and indications relative to the timing of regulatory approval and publication of key randomized trials. METHODS: We performed a retrospective cohort study using the OptumLabs Data Warehouse of US commercial insurance enrollees from 2006 to 2019. We quantified PFO closure among individuals with ≥2 years of preprocedure coverage to establish indications, classified hierarchically as stroke/systemic embolism, migraine, transient ischemia attack, or other. RESULTS: We identified 5315 patients undergoing PFO closure (51.8% female, 29.2%≥60 years old), which increased from 4.75 per 100 000 person-years in 2006 to 6.60 per 100 000 person-years in 2019. Patients aged ≥60 years accounted for 29.2% of closures. Procedure volumes corresponded weakly with supportive clinical publications and device approval. Among patients with PFO closure, 58.6% underwent closure for stroke/systemic embolism, 10.2% for transient ischemia attack, 8.8% for migraine, and 22.4% for other indications; 17.6% of patients had atrial fibrillation at baseline; and 11.9% developed atrial fibrillation postprocedure. Those aged ≥60 years and male were less likely to undergo closure for migraine than stroke/systemic embolism. CONCLUSIONS: From 2006 to 2019, PFO closure use was consistently low and corresponded weakly with clinical trial publications and regulatory status. Nearly half of patients underwent PFO closure for indications unapproved by the Food and Drug Administration. Regulators and payers should coordinate mechanisms to promote utilization for approved indications to ensure patient safety and should facilitate clinical trials for other possible indications.

Early technology review: towards an expedited pathway.

OBJECTIVES: Evidence development for medical devices is often focused on satisfying regulatory requirements with the result that health professional and payer expectations may not be met, despite considerable investment in clinical trials. Early engagement with payers and health professionals could allow companies to understand these expectations and reflect them in clinical study design, increasing chances of positive coverage determination and adoption into clinical practice. METHODS: An example of early engagement through the EXCITE International model using an early technology review (ETR) is described which includes engagement with payers and health professionals to better inform companies to develop data that meet their expectations. ETR is based on an early evidence review, a framework of expectations that guides the process and identified gaps in evidence. The first fourteen ETRs were reviewed for examples of advice to companies that provided additional information from payers and health professionals that was thought likely to impact on downstream outcomes or strategic direction. Given that limitations were imposed by confidentiality, examples were genericized. RESULTS: Advice through early engagement can inform evidence development that coincides with expectations of payers and health professionals through a structured, objective, evidence-based approach. This could reduce the risk of business-related adverse outcomes such as failure to secure a positive coverage determination and/or acceptance by expert health professionals. CONCLUSIONS: Early engagement with key stakeholders exemplified by the ETR approach offers an alternative to the current approach of focusing on regulatory expectations. This could reduce the time to reimbursement and clinical adoption and benefit patient outcomes and/or health system efficiencies.

Developing minimum core data structure for the obesity devices Coordinated Registry Network (CRN).

Obesity continues to be a major public health issue, with more than two-thirds of adults in the USA categorized as overweight or obese. Bariatric surgery is effective and yields durable weight loss; however, few qualified candidates choose to undergo surgical treatment. Less-invasive alternatives to bariatric surgery are being developed to bridge the treatment gap. Recognizing the burden of conducting pivotal clinical trials and traditional post-approval studies for medical devices, the Food and Drug Administration (FDA) Center for Devices and Radiological Health has encouraged the development of real-world data content and quality that is sufficient to provide evidence for Total Product Life Cycle medical device evaluation. A key first step is to establish a minimum core data structure that provides a common lexicon for endoscopic obesity devices and its corresponding interoperable data elements. Such a structure would facilitate data capture across existing workflow with a 'coordinated registry network' capability. On July 29, 2016, a workshop entitled, 'GI Coordinated Registry Network: A Case for Obesity Devices' was held at the FDA White Oak Campus by the Medical Device Epidemiology Network public-private partnership and FDA to initiate the work of developing a common lexicon and core data elements in the metabolic device space, which marked the inauguration of the Gastrointestinal Coordinated Registry Network project. Several work groups were subsequently formed to address clinical issues, data quality issues, registry participation, and data sharing.

Use of real-world data and clinical registries to identify new uses of existing vascular endografts: combined use of GORE EXCLUDER Iliac Branch Endoprosthesis and GORE VIABAHN VBX Balloon Expandable Endoprosthesis.

Objective: To assess the feasibility of collecting, examining and reporting observational, real-world evidence regarding the novel use of the GORE EXCLUDER Iliac Branch Endoprosthesis (IBE) in conjunction with the GORE VIABAHN VBX Balloon Expandable Endoprosthesis (IBE+VBX stent graft). Design: Multicentre retrospective cohort study. Setting: Four real-world data sources were used: a national quality improvement registry, a statewide clinical research network, a regional quaternary health system and two tertiary academic medical centres. Participants: In total, 30 patients with 37 IBE+VBX stent graft were identified. Of those, the mean age was 72±10.2 years and 90% were male. The cohort was 77% white, 10% black, 3% Hispanic and 10% other. Main outcome measures: Outcome measures included: proportion of percutaneous vs open surgical access, intensive care admission, intensive care unit (ICU) length-of-stay (LOS), total LOS, postoperative complications, discharge disposition and 30-day mortality. Results: The majority (89%) of cases were performed percutaneously, 5% required surgical exposure following failed percutaneous access and 6% required open surgical exposure outright. Nearly half (43%) required intensive care admission with a median ICU LOS of 1 day (range: 1-2). Median total LOS was 1 day (IQR: 1-2). There were zero postoperative myocardial infarctions, zero reported leg embolisations and no reported reinterventions. Access site complications were described in 1 of 28 patients, manifesting as a haematoma or pseudoaneurysm. Ultimately, 97% were discharged to home and one patient was discharged to a nursing home or rehabilitation facility. There were no 30-day perioperative deaths. Conclusions: This project demonstrates the feasibility of identifying and integrating real-world evidence, as it pertains to an unapproved combination of endovascular devices (IBE+VBX stent graft), for short-term outcomes analysis. This new paradigm of evidence has potential to be used for device monitoring, submission to regulatory agencies, or consideration in indication expansions and approvals with further efforts to systematise data collection and transmission mechanisms.

[Practical use of digital health applications (DiGA) in internal medicine]. / Praktische Anwendung digitaler Gesundheitsanwendungen (DiGA) in der Inneren Medizin.

Since 2020 physicians can prescribe digital health applications (DiGA), also colloquially known as apps on prescription, which are reimbursed by the statutory health insurance when they are approved by the Federal Institute for Drugs and Medical Devices (BfArM) and are included in the DiGA Ordinance. Currently, there is one approved DiGA (indication obesity) for internal medicine. There are many questions on the practical use of the DiGA, ranging from the prescription, the effectiveness, the complexities and reimbursement as well as the liability risks. The DiGA are innovative new means, which maybe support internal medicine physicians in the diagnostics and treatment in the future. The benefits in this field of indications are limited by unclarified issues, especially on the prescription practice and the currently low number of DiGA available in internal medicine.

Bringing hydrogel-based craniofacial therapies to the clinic.

This review explores the evolution of the use of hydrogels for craniofacial soft tissue engineering, ranging in complexity from acellular injectable fillers to fabricated, cell-laden constructs with complex compositions and architectures. Addressing both in situ and ex vivo approaches, tissue restoration secondary to trauma or tumor resection is discussed. Beginning with relatively simple epithelia of oral mucosa and gingiva, then moving to more functional units like vocal cords or soft tissues with multilayer branched structures, such as salivary glands, various approaches are presented toward the design of function-driven architectures, inspired by native tissue organization. Multiple tissue replacement paradigms are presented here, including the application of hydrogels as structural materials and as delivery platforms for cells and/or therapeutics. A practical hierarchy is proposed for hydrogel systems in craniofacial applications, based on their material and cellular complexity, spatial order, and biological cargo(s). This hierarchy reflects the regulatory complexity dictated by the Food and Drug Administration (FDA) in the United States prior to commercialization of these systems for use in humans. The wide array of available biofabrication methods, ranging from simple syringe extrusion of a biomaterial to light-based spatial patterning for complex architectures, is considered within the history of FDA-approved commercial therapies. Lastly, the review assesses the impact of these regulatory pathways on the translational potential of promising pre-clinical technologies for craniofacial applications. STATEMENT OF SIGNIFICANCE: While many commercially available hydrogel-based products are in use for the craniofacial region, most are simple formulations that either are applied topically or injected into tissue for aesthetic purposes. The academic literature previews many exciting applications that harness the versatility of hydrogels for craniofacial soft tissue engineering. One of the most exciting developments in the field is the emergence of advanced biofabrication methods to design complex hydrogel systems that can promote the functional or structural repair of tissues. To date, no clinically available hydrogel-based therapy takes full advantage of current pre-clinical advances. This review surveys the increasing complexity of the current landscape of available clinical therapies and presents a framework for future expanded use of hydrogels with an eye toward translatability and U.S. regulatory approval for craniofacial applications.

Data Safety and Monitoring Boards Should Be Required for Both Early- and Late-Phase Clinical Trials.

Phase I and II clinical trials increasingly combine therapeutic and toxicity endpoints. Recently, therapeutic agents have even achieved U.S. Food and Drug Agency approval based on early phase trials alone. These developments point to new challenges in assuring the safety of human research subjects and patients. Given their size and use of real-world patients, phase III studies warrant independent monitoring by a Drug Safety Monitoring Board (DSMB). Requirements should also be extended to include many phase I and II clinical trials. Measures should be taken to establish and standardize minimum qualifications for service on a DSMB.

Registration, publication, and outcome reporting among pivotal clinical trials that supported FDA approval of high-risk cardiovascular devices before and after FDAAA.

BACKGROUND: Selective registration, publication, and outcome reporting of clinical trials distort the primary clinical evidence that is available to patients and clinicians regarding the safety and efficacy of US Food and Drug Administration (FDA)-approved medical devices. The purpose of this study is to compare registration, publication, and outcome reporting among pivotal clinical trials that supported FDA approval of high-risk (class III) cardiovascular devices before and after the FDA Amendment Act (FDAAA) was enacted in 2007. METHODS: Using publicly available data from ClinicalTrials.gov , FDA summaries, and PubMed, we determined registration, publication, and reporting of findings for all pivotal clinical studies supporting FDA approval of new high-risk cardiovascular devices between 2005 and 2020, before and after FDAAA. For published studies, we compared both the primary efficacy outcome with the FDA's Premarket Approval (PMA) primary efficacy outcome and the published interpretation of findings with the FDA reviewer's interpretation (positive, equivocal, or negative). RESULTS: Between 2005 and 2020, the FDA approved 156 high-risk cardiovascular devices on the basis of 165 pivotal trials, 48 (29%) of which were categorized as pre-FDAAA and 117 (71%) as post-FDAAA. Post-FDAAA, pivotal clinical trials were more likely to be registered (115 of 117 (98%) vs 24 of 48 (50%); p < 0.001), to report results (98 of 117 (87%) vs 7 of 48 (15%); p < 0.001) on ClinicalTrials.gov , and to be published (100 or 117 (85%) vs 28 of 48 (58%); p < 0.001) in peer-reviewed literature when compared to pre-FDAAA. Among published trials, rates of concordant primary efficacy outcome reporting were not significantly different between pre-FDAAA trials and post-FDAAA trials (24 of 28 (86%) vs 96 of 100 (96%); p = 0.07), nor were rates of concordant trial interpretation (27 of 28 (96%) vs 93 of 100 (93%); p = 0.44). CONCLUSIONS: FDAAA was associated with increased registration, result reporting, and publication for trials supporting FDA approval of high-risk medical devices. Among published trials, rates of accurate primary efficacy outcome reporting and trial interpretation were high and no different post-FDAAA.

Mulheres na gestão de tecnologias e engenharia clínica: o caso dos ventiladores pulmonares na Covid-19 / Women in technology management and clinical engineering: the case of pulmonary ventilators at Covid-19

RESUMO O risco iminente de desabastecimento de ventiladores pulmonares nos serviços de saúde acarretou diversas frentes de trabalho para disponibilizar o maior número possível desses equipamentos para o tratamento dos pacientes acometidos. O agravamento da crise sanitária colapsou serviços de saúde com busca isocrônica por leitos. Em meio ao colapso, foi detectada nova variante da linhagem Sars-CoV-2 e confirmado o primeiro caso de reinfecção. Entre os pontos críticos, foi destaque a escassez caótica de oxigênio e taxas de ocupação de leitos acima de 90%. Pretende-se relatar a participação das autoras nas iniciativas para o enfrentamento da pandemia de relevância internacional. Destacar a participação de mulheres em atividades que são cruciais para responder, em tempo oportuno, às demandas oriundas de emergências sanitárias. Por meio de método de pesquisa de abordagem descritiva e exploratória, buscou-se verificar o perfil das mulheres atuantes na gestão de tecnologias no enfrentamento da pandemia. Destaca-se a pesquisa recente da Associação Brasileira de Engenharia Clínica que verificou que apenas 19% dos associados respondentes eram mulheres, enquanto 81% eram do sexo masculino. Assim, divulgar e dar amplo conhecimento das ações de mulheres nessa área pode colaborar para o alcance da igualdade de gênero e empoderar todas as mulheres e meninas.

ABSTRACT The imminent risk of shortage of pulmonary ventilators in health services has resulted in several work fronts to maintain and make available the largest possible number of equipment available for the treatment of patients. The worsening of the health crisis has collapsed health services with an isochronic search for beds. Amid the collapse, a new variant of the Sars-CoV-2 strain was detected and the first case of reinfection was confirmed. Among the critical points was the chaotic oxygen scarcity and bed occupancy rates above 90%. We intend to report the participation of the authors in the initiatives to face the pandemic; highlight the participation of women in activities that are crucial to respond, in a timely manner, to the demands arising from health emergencies. Through a research method with a descriptive and exploratory approach, we sought to verify the profile of women working in the management of technologies in facing the pandemic. The recent survey by the Brazilian Association of Clinical Engineering stands out, which found that only 19% of the respondent associates were women. Thus, disseminating and giving broad knowledge of women´s actions in this area can collaborate in achieving gender equality and empower all women and girls.

Artificial intelligence in radiology: 100 commercially available products and their scientific evidence.

OBJECTIVES: Map the current landscape of commercially available artificial intelligence (AI) software for radiology and review the availability of their scientific evidence. METHODS: We created an online overview of CE-marked AI software products for clinical radiology based on vendor-supplied product specifications ( www.aiforradiology.com ). Characteristics such as modality, subspeciality, main task, regulatory information, deployment, and pricing model were retrieved. We conducted an extensive literature search on the available scientific evidence of these products. Articles were classified according to a hierarchical model of efficacy. RESULTS: The overview included 100 CE-marked AI products from 54 different vendors. For 64/100 products, there was no peer-reviewed evidence of its efficacy. We observed a large heterogeneity in deployment methods, pricing models, and regulatory classes. The evidence of the remaining 36/100 products comprised 237 papers that predominantly (65%) focused on diagnostic accuracy (efficacy level 2). From the 100 products, 18 had evidence that regarded level 3 or higher, validating the (potential) impact on diagnostic thinking, patient outcome, or costs. Half of the available evidence (116/237) were independent and not (co-)funded or (co-)authored by the vendor. CONCLUSIONS: Even though the commercial supply of AI software in radiology already holds 100 CE-marked products, we conclude that the sector is still in its infancy. For 64/100 products, peer-reviewed evidence on its efficacy is lacking. Only 18/100 AI products have demonstrated (potential) clinical impact. KEY POINTS: • Artificial intelligence in radiology is still in its infancy even though already 100 CE-marked AI products are commercially available. • Only 36 out of 100 products have peer-reviewed evidence of which most studies demonstrate lower levels of efficacy. • There is a wide variety in deployment strategies, pricing models, and CE marking class of AI products for radiology.

Key Principles of Clinical Validation, Device Approval, and Insurance Coverage Decisions of Artificial Intelligence.

Artificial intelligence (AI) will likely affect various fields of medicine. This article aims to explain the fundamental principles of clinical validation, device approval, and insurance coverage decisions of AI algorithms for medical diagnosis and prediction. Discrimination accuracy of AI algorithms is often evaluated with the Dice similarity coefficient, sensitivity, specificity, and traditional or free-response receiver operating characteristic curves. Calibration accuracy should also be assessed, especially for algorithms that provide probabilities to users. As current AI algorithms have limited generalizability to real-world practice, clinical validation of AI should put it to proper external testing and assisting roles. External testing could adopt diagnostic case-control or diagnostic cohort designs. A diagnostic case-control study evaluates the technical validity/accuracy of AI while the latter tests the clinical validity/accuracy of AI in samples representing target patients in real-world clinical scenarios. Ultimate clinical validation of AI requires evaluations of its impact on patient outcomes, referred to as clinical utility, and for which randomized clinical trials are ideal. Device approval of AI is typically granted with proof of technical validity/accuracy and thus does not intend to directly indicate if AI is beneficial for patient care or if it improves patient outcomes. Neither can it categorically address the issue of limited generalizability of AI. After achieving device approval, it is up to medical professionals to determine if the approved AI algorithms are beneficial for real-world patient care. Insurance coverage decisions generally require a demonstration of clinical utility that the use of AI has improved patient outcomes.

Emerging Technologies for Identifying Atrial Fibrillation.

Atrial fibrillation (AF) is a major cause of morbidity and mortality globally, and much of this is driven by challenges in its timely diagnosis and treatment. Existing and emerging mobile technologies have been used to successfully identify AF in a variety of clinical and community settings, and while these technologies offer great promise for revolutionizing AF detection and screening, several major barriers may impede their effectiveness. The unclear clinical significance of device-detected AF, potential challenges in integrating patient-generated data into existing healthcare systems and clinical workflows, harm resulting from potential false positives, and identifying the appropriate scope of population-based screening efforts are all potential concerns that warrant further investigation. It is crucial for stakeholders such as healthcare providers, researchers, funding agencies, insurers, and engineers to actively work together in fulfilling the tremendous potential of mobile technologies to improve AF identification and management on a population level.